Knowledge Transfer Graph for Deep Collaborative Learning

Knowledge transfer among multiple networks using their outputs or intermediate activations have evolved through manual design from a simple teacher-student approach to bidirectional cohort one. The major components of such knowledge framework involve the network size, number networks, direction, and loss function. However, because these factors are enormous when combined become intricately entangled, methods conventional explored only limited combinations. In this paper, we propose novel graph representation called that provides unified view has potential represent diverse patterns. We also four gate functions control gradient can deliver combinations transfer. Searching structure enables us discover more effective than manually designed Experimental results show proposed method achieved performance improvements.